Regression is used in many applications to predict the continuous value of an output, such as the value of the stock market or the pixel intensity in an image, given a new input. Regression uses training data which is a collection of observed input and output pairs to perform the prediction. Probabilistic regression provides a degree of uncertainty in addition to the prediction of an output. The degree of uncertainty provides an indication of the confidence associated with the predicted output value and this may be very useful in decision making. For example, a different decision may be made if the regression indicates a low confidence in a value compared to a high confidence in the same value.
There are a number of known techniques for performing accurate probabilistic regression; however, all these techniques have a high computational cost of learning from data and of making predictions. This means that they are not suitable for many applications; in particular they are not suitable for applications where decisions need to be made quickly. A number of techniques have been proposed to make probabilistic regression more efficient and these are based on sparse linear models. Sparse linear models use linear combinations of a reduced number of basis functions. These sparse linear models are, however, not suitable for use in decision making because they are overconfident in their predictions, particularly in regions away from any training data.
The embodiments described below are not limited to implementations which solve any or all of the disadvantages of known techniques for performing probabilistic regression.